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Abstract — A new multiple classifier approach for spectral- 
spatial classification of hyperspectral images is proposed. Several 
classifiers are used independently to classify an image. For 
every pixel, if all the classifiers have assigned this pixel to the 
same class, the pixel is kept as a marker, i,e., a seed of the 
spatial region, with the corresponding class label. We propose 
to use spectral-spatial classifiers at the preliminary step of the 
marker selection procedure, each of them combining the results 
of a pixehvise classification and a segmentation map. Different 
segmentation methods based on dissimilar principles lead to 
different classification results. Furthermore, a minimum spanning 
forest is built, where each tree is rooted on a classification-driven 
marker and forms a region in the spectral-spatial classification 
map. Experimental results are presented for two hyperspectral 
airborne images. The proposed method significantly improves 
classification accuracies, when compared to previously proposed 
classification techniques. 

Index Terms — Hyperspectral images, classification, segmenta- 
tion, multiple classifiers, minimum spanning forest 

I. Introduction 

Hyperspectral imaging is a relatively recent technique in 
remote sensing, Acquired remotely by airborne or space- 
bome sensors, hyperspectral data are comprised of hundrends 
of spatially co-registered images corresponding to different 
spectral channels [1], [2]. Figure 1 illustrates the structure 
of a hyperspectral image. Every pixel is presented as a B- 
dimensional feature vector across the wavelength dimension, 
called the spectrum of the material in this pixel. This rich 
information in every spatial location increases the capability 
to distinguish different physical materials. Thus, hyperspec- 
tral imagery opens new perspectives for image classification, 
which is an important task for a wide variety of applications 
(precision agriculture, monitoring and management of the 
environment, security issues). 

However, such a large number of spectral channels, usually 
coupled with limited availability of reference data presents 

This research is supported in part by the Marie Curie Research Training 
Network “'HYPER 4-NUT’. 

1 By reference data, we mean manually labeled pixels which are used for 
training classifiers followed by assessment of classification accuracies. 



Fig. 1 . Structure of a hyperspectral image. 


challenges to image analysis. While pixelwise classification 
techniques process each pixel independently without consid- 
ering information about spatial structures [3], [4], [5], [6], 
further improvement of classification results can be achieved 
by considering spatial dependencies between pixels, i.e., by 
perfonning spectral-spatial classification [7], [8], [9], [10], 
[11], [12], 

Segmentation techniques, partitioning an image into homo- 
geneous regions with respect to some criterion of interest 
(called homogeneity criterion, e.g., intensity or texture), are 
powerful tools for defining spatial dependencies [13]. In 
previous works, we have distinguished spatial structures in 
the hyperspectral image by perfonning unsupervised segmen- 
tation [12], [14], [15]. Watershed, partitional clustering and Hi- 
erarchical SEGmentation (HSEG) techniques have been used 
for this purpose. Segmentation and pixelwise classification 
were applied independently, then results were combined using 
a majority voting rule (see Figure 2). Thus, every region from 
a segmentation map was considered as an adaptive homo- 
geneous neighborhood for all the pixels within this region, 
The described technique led to a signification improvement 
of classification accuracies and provided more homogeneous 
classification maps, when compared to classification tech- 
niques using local, neighborhoods in order to include spatial 
information into a classifier. 



An alternative way to get accurate segmentation results 
consists in performing a marker-controlled segmentation f 13], 
[16]. The idea behind this approach is to select for every spatial 
object one or several pixels belonging to this object (called a 
region seed, or a marker of the corresponding region) and to 
grow regions from the selected seeds, so that every region in 
the resulting segmentation map is associated with one region 
seed. The markers of regions can be chosen either manually, or 
automatically. Recently we have proposed to use probability 
estimates obtained by the pixelwise Support Vector Machines 
(SVM) classification in order to select the most reliable 
classified pixels as markers, i.e., seeds of spatial regions [17], 
Furthermore, image pixels were grouped into a Minimum 
Spanning Forest (MSF), where each tree was rooted on a 
classification-derived marker. The decision to connect a pixel, 
which was not yet in the forest, to one of the trees in the forest 
was based on its similarity to one of the adjacent pixels already 
belonging to the forest. By assigning the. class of each marker 
to all the pixels within the region grown from this marker, a 
spectral-spatial classification map was obtained. The described 
technique led to a significant improvement of classification 
accuracies when compared to previously proposed methods. 
The drawback of this method is that the choice of markers 
strongly depends on the performances of the selected pixelw ise 
classifier (e.g., the SVM classifier in our previous work [17]). 

In this work, we aim to mitigate the dependence of the 
marker selection procedure from the choice of a pixelwise clas- 
sifier. This can be achieved by using not a single classification 
algorithm for marker selection, but an ensemble of classifiers, 
i.e., multiple classifiers. For this purpose, several individual 
classifiers must be chosen and combined within one system in 
such a way that the complementary benefits of each classifier 
are used, while their weaknesses are avoided. 

In this paper, a new marker selection method based on a 
multiple classifier (MC) system is proposed. Several classifiers 
are used independently to classify an image. Furthermore, a 
marker map is constructed by selecting the pixels assigned 
by all the classifiers to the same class. We propose to use 
spectral-spatial classifiers in the preliminary step of the marker 
selection procedure, each of them combining the results 
of a pixelwise classification and one of the unsupervised 
segmentation techniques (see Figure 2). By using spectral- 
spatial classifiers in this step, spatial context in the image is 
taken into account, and classification maps are more accurate 
when compared to pixelwise classification maps. This leads to 
more accurate marker selection results. The proposed marker 
selection method is incorporated into a new Multiple Spectral- 
Spatial Classification (MSSC) scheme (MSSC-MSF) based 
on the construction of an MSF from region markers. 

In order to assess the importance of spectral-spatial ap- 
proaches for marker selection, we have also implemented a 
Multiple Classification scheme ( MC-MSF). Here, spectral- 
spatial classification maps are replaced by the maps obtained 
using pixelwise classification techniques. Finally, a marker 
map is computed and an MSF from the selected markers is 
constructed. 

Although the classification approach proposed in this paper 
has been designed for hyperspectral data, the method is general 
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Fig. 2. Example of speetral-spaliai classification using majority voting within 
segmentation regions. 


and can be applied for other types of data as well. Two hyper- 
spectral airborne images are used to demonstrate experimental 
results: an image recorded by the Reflective Optics System 
Imaging Spectrometer (ROSIS) over the University of Pavia, 
Italy, and an Airborne Visible/Infrared imaging Spectrometer 
(AVIRIS) image acquired over Northwestern Indiana’s Indian 
Pines site [ 1 Sj. 

The paper is organized as follows. In the next section, the 
multiple classifier approach is briefly discussed. Section III 
describes the proposed classification scheme. Experimental 
results are discussed in Section IV, Finally, conclusions are 
drawn in Section V. 

II. Multiple Classifier Approach 

The traditional approach for a pattern recognition problem 
is to search for the individual algorithm giving the best 
possible classification performances. However, in many cases, 
the classification accuracy can be improved by using an 
ensemble of classifiers, or multiple classifiers. This is due 
to the fact that although one of the classification algorithms 
would yield the best performances, the sets of pixels (patterns 
in general) misclassified by the different algorithms would not 
necessarily overlap. Thus, the aim of an MC system is to 
determine an efficient combination method that makes use of 
the complementary benefits of each classifier, while tackling 
the individual, drawbacks [19], [20], [21], 
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Fig. 4. Schematic diagram of a multiple classifier system. 

A schematic representation of an MC system is given in 
Figure 4. An important issue for an efficient MC system is 
that the individual classifiers should be independent. More 
precisely, the classifiers should not agree with each other when 
they misclassify a pixel [20]. The complementary properties 
of the different classifiers selected for the MC system should 
ensure to a certain extent this requirement. 

Another important issue is the rule for combining the indi- 
vidual classifiers {i.e., combination function). The individual 
classifier outputs, such as class labels and possibly posterior 
probabilities, are typically combined by voting rules, belief 
functions, statistical techniques, the Dempster-Shafer evidence 
theory, and other schemes [19]. For a given pixel, if all the 
classifiers agree on the same class k, the evident combination 
rule consists in assigning this pixel to the class k in the 
final classification map. On the other side, when individual 
classifiers disagree in assigning the given pixel, the procedure 
of final decision making is not that straightforward, and 
different combination functions may yield different results, A 
typical result of the MC system is a final classification map, 
where each pixel has a unique class label. This type of MC 
systems has been previously used for remote sensing image 
classification [21], [22], [23], 

In this paper, we propose to address the combination rule 
issue in the following way: According to the exclusionary rule, 


only the pixels where all the classifiers agree, i.e., the most 
reliable pixels, are kept in the classification map. The rest 
of the pixels are further classified by constructing an MSF 
rooted on the “reliable” pixels, i.e., by incorporating the spatial 
information into classification. 

Coming back to the first issue for designing an MC system, 
different individual classifiers must be chosen. For instance, 
standard pixetwise classification algorithms can be be used 
for this purpose, such as SVM, Maximum Likelihood (ML), 
k-Nearest Neighborhood (k-NN) methods (parametric and non- 
parametric techniques, based on different principles). We have 
used these individual techniques in the MC-MSF classification 
scheme. Furthermore, we propose to use spectral-spatial clas- 
sifiers as individual classifiers for the MC system (MSSC-MSF 
classification scheme), each of them combining the results of 
a pixelwise classification and one of the unsupervised seg- 
mentation techniques. Different segmentation methods based 
on dissimilar principles lead to different classification results. 
The use of spectral-spatial classifiers yields more accurate 
classification maps, when compared to those obtained by 
performing pixelwise classification. 


III. Proposed Classification Scheme 

The flow-chart of the proposed MSSC-MSF classification 
method is depicted in Figure 3. At the input a i3-band 
hyperspectral image is given, which can be considered as 
a set of n pixel vectors X = {ij € 1 I s , j = 1.2, ...,n). 
Classification consists in assigning each pixel to one of the K 
classes of interest. In the following, each step of the proposed 
procedure is described. 

Segmentation can be defined as an exhaustive partitioning 
of the input image into regions, each of which is considered to 
be homogeneous with respect to some criterion of interest. We 
have investigated the use of three techniques for hyperspectral 
image segmentation, as described hereafter. 
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A. Watershed segmentation 

Watershed transformation is a powerful morphological ap- 
proach to image segmentation which combines region growing 
and edge detection. The watershed is usually applied to the 
gradient function, and it divides an image into regions, so that 
each region is associated with one minimum of the gradient 
image [24], 

The extension of a watershed technique to the case of 
hyperspectral images has been investigated in [15], [25], In 
this paper, we present watershed results, obtained by the 
scheme we proposed and described in [35]: First, a one-band 
Robust Color Morphological Gradient (RCMG) [26] for the 
hyperspectral image is computed. 

For each pixel vector x p , let x — .... xf] be a set of 

e vectors contained within a structuring element £ (i.e., the 
pixel x p itself and e. — 1 neighboring pixels). A 3x3 square 
structuring element with the origin in its center is typically 
used. The Color Morphological Gradient (CMG), using the 
Euclidean distance, is computed as: 

CM Ge (x p ) = max { |j x‘ - x 1 S}. ( 3 ) 

i.e., the maximum of the distances between all pairs of vectors 
in the set x ■ One of the drawbacks of the CMG is that it is very 
sensitive to noise. To overcome the problem of outliers, the 
RCMG has been proposed [26]. The scheme to make a CMG 
robust consists of removing the two pixels that are furthest 
apart and then finding the CMG of the remaining pixels. This 
process can be repeated several times until a good estimate of 
the gradient is obtained. 

Thus, the RCMG , using the Euclidean distance, can be 
defined as: 

RCMGe(\ p ) - max {|lx‘ - x£|| 2 }, (2) 

i,j€{x-REM r ] r 1 

where REM r is a set of r vector pairs removed. If E is a 
3x3 square structuring element, r — 1 is recommended [26], 

Furthermore, watershed transformation is applied on the 
gradient image, using a standard algorithm [27], As a result, 
the image is partitioned into a set of regions, and one subset of 
watershed pixels, i.e,, pixels situated on the borders between 
regions. Finally, every watershed pixel is assigned to the 
neighboring region with the “closest” median 2 (the distance 
between the vector median of this region and the watershed 
pixel is minimal). 

B. Segmentation by expectation maximization 

The Expectation Maximization (EM) algorithm for the 
Gaussian mixture resolving belongs to the group of partitional 
clustering techniques [14], [29]. The use of partitional cluster- 
ing for hyperspectral image segmentation has been discussed 
in [14], Clustering aims at finding groups of spectrally similar 
pixels. We assume that pixels belonging to the same cluster are 
drawn from a multivariate Gaussian probability distribution. 

2 A standard vector median [28] for the region S — £ E®,j = 

1,2,.,.,/} is defined as sy-.yf = arg fnin s , s { II s s, ]- 


Each image pixel can be statistically modelled by the following 
probability density fiinction: 

c 

pOO = 51 ^ v ' 2 ' ( x > s =) (3) 

c--l 

where C is the number of clusters, te c g [0, 1] is the mixing 
proportion (weight) of duster c with = 1, and 

S) is the multivariate Gaussian density with mean p 
and covariance matrix E: 

d> c (x: p c , S,- ; ) = 

11 

n£)B /2 jsTj 1/2 exp{-4{x-// f; ) T E r 7 l (x-/xJ}. (4) 

The parameters of the distributions tp ----- (C, u C! fi cl E c ; c: = 
1,2, ...,C} are estimated by the EM algorithm, as described 
in [14]. An upper bound on the number of clusters, which 
is a required input parameter, is recommended to be chosen 
slightly superior to the number of classes. 

When the algorithm converges, the partitioning of the set 
of image pixels into clusters is obtained. However, as no 
spatial information is used during the clustering procedure, 
pixels with the same duster label can form a connected 
spatial region, or can belong to disjoint regions. In order to 
obtain a segmentation map, a connected components labeling 
algorithm [30] is applied to the output image partitioning 
obtained by clustering. 

The total number of parameters to be estimated by the EM 
algorithm is P = {B(B + l)/2 + B 4- 1)C -F 1, where B is 
a dimensionality of feature vectors. If the value of B is large, 
P may be quite a large number. This may cause the problem 
of the covariance matrix singularity or inaccurate parameter 
estimation results. In order to avoid these problems, we pro- 
pose to previously apply a feature reduction, using the method 
of piecewise constant function approximations ( PCFA ) [31], 
which has shown good performances for hyperspectral data 
feature extraction. 

C. RHSEG segmentation 

The Hierarchical image SEGmentatton (HSEG) algorithm 
is a segmentation technique based on iterative hierarchical 
step-wise optimization region growing method. Furthermore, 
it provides a possibility of merging non-adjacent regions by 
spectral clustering [32]. 

The following outline of the HSEG algorithm is based on 
the description given in [33], [32]: 

1) Initialize the segmentation by assigning for each pixel 
a region label. If a pre-segmentation is provided, label 
each pixel according to the pre-segmentation. Otherwise, 
label each pixel as a separate region. 

2) Calculate the dissimilarity criterion value between all 
pairs of spatially adjacent regions. 

3) Find the smallest dissimilarity criterion value 
dissim_val and set thresh_val equal to it. Then 
merge ail pairs of spatially adjacent regions with 
dissimjeal — threshjual. 
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4) If a parameter S wg tn > 0-0, merge alt pairs of spa- 
tially non-adjacent regions with dissim_pal < S wg ht - 
threshjuat. 

5) If convergence is not achieved, go to step (2). 

In order to reduce computational demands, a Recursive 
divide-and-conquer approximation of HSEG (RHSEG) has 
been developed, The NASA-Goddard RHSEG software pro- 
vides an efficient implementation of the RHSEG algorithm. 

When determining most similar pair of regions, we propose 
to choose the standard Spectral Angle Mapper (SAM) between 
the region mean vectors and as the dissimilarity criterion [32], 
The SAM measure between u 4 and u , (u*,Uj 6 8 s ) 
determines the spectral similarity between two vectors by 
computing the angle between them. It is defined as 


SAM (u; , Uj ) = arccos 


fe=l u ib U jb 




^] !/2 


( 5 ) 


The optional parameter S wg h,t tunes the relative importance 
of spectral clustering versus region growing. If S wg ht = 0.0. 
only merging of spatially adjacent regions is performed. If 
0.0 < S wg ht 5 TO, merging between spatially adjacent 
regions is favored compared to merging of spatially non- 
adjacent regions by a factor of 1.0 /S wg ht- As discussed 
in [34], the optimal parameter S wg ut can be chosen based on 
a priori knowledge about information classes contained in the 
image. If some classes have very similar spectral responses, 
we recommended to choose S wg ht — 0.0 or close to this 
value 3 . Otherwise, we recommend increasing the possibility 
of merging spatially non-adjacent regions. If S v;g ht > 0.0, 
labeling of connected components has to be applied after 
RHSEG in order to obtain a segmentation map where each 
spatially connected component has a unique label. 

RHSEG provides as output a hierarchical sequence of 
image partitions, In this sequence, a particular object can 
be represented by several regions at finer levels of details, 
and can be assimilated with other objects in one region at 
coarser levels of details. This hierarchical sequence allows 
flexibility in choosing the appropriate level of detail for the 
segmentation map. When training data is available, it is a 
simple process to quantitatively evaluate the segmentation 
results at each hierarchical level versus the training data to 
select the appropriate level of detail. Otherwise an appropriate 
level of segmentation detail can be chosen interactively with 
the program HSEGViewer [32], or an automated method, 
tailored to the application, can be developed such as explored 
in [35], 


D. Pixelwise classification 

Independent of the previous steps, a pixelwise classification 
of the hyperspectral image is performed. We propose to use 
an SVM classifier for this purpose. Other pixelwise classifiers 
could be used. However, SVM perform extremely well in 

’The analysis reported in this paper was performed with version 1.40 of 
the RHSEG software. The recently released version 1,50 of RHSEG produces 
similar segmentation results, except that it can exhibit improved results for 
larger values of S, Pg ht especially for data sets containing classes with mostly 
dissimilar spectral responses. 


classifying high-dimensional data when a limited number of 
training samples are available [5], [36], We refer the reader to 
[5], [37] for details on SVM technique. This step results in a 
classification map, where each pixel has a unique class label. 

E. Majority voting within segmentation regions 

Each of the obtained unsupervised segmentation maps is 
combined with the pixelwise classification map using the 
majority voting principle: For every region in the segmentation 
map, all the pixels are assigned to the most frequent class 
within this region (see an illustrative example in Figure 2). 
Thus, q segmentation maps combined with the pixelwise 
classification map result in q spectral-spatial classification 
maps (since we propose to use three different segmentation 
techniques, in this particular case q = 3). 


F. Marker selection 

This step consists of computing a map of markers, using 
spectral-spatial classification maps from the previous step and 
exclusionary rule: For every pixel, if all the classifiers agree, 
the pixel is kept as a marker, with the corresponding class 
label. The resulting map of rn markers contains the most 
reliably classified pixels. 


G. Construction of a Minimum Spanning Forest 

In the final step, image pixels are grouped into an MSF 
rooted on the selected markers [17]. Each pixel is considered 
as a vertex v € V of an undirected graph G — (V, E, W ), 
where V and E are the sets of vertices and edges, respectively, 
and W is a mapping of the set of the edges E into K + . Each 
edge e.ij e E of this graph connects a couple of vertices i 
and j corresponding to the neighboring pixels. Furthennore, 
a weight utij is assigned to each edge dj, which indicates 
the degree of dissimilarity between two vertices (i.e,, two 
corresponding pixels) connected by this edge. We propose to 
use an 8-neighborhood and the SAM measure for computing 
weights of edges, as described in [17], 

Given a graph G = ( V , E, FF). the MSF rooted on a set of 
m distinct vertices {t\. consists in finding a spanning 
forest F* — (V. Ep- ) of G , such that each distinct tree of F* 
is grown from one root t t , and the sum of the edges weights 
of F * is minimal [38] 


F' £ arg min 
FeSF 



(6) 


where SF is a set of all spanning forests of G rooted on 

In order to obtain the MSF rooted on rn markers cor- 
responding to the vertices f,:J = an additional 

root vertex r is added and is connected by the null-weight 
edges to the vertices l,. The minimum spanning tree of the 
constructed graph induces an MSF in G, where each tree is 
grown on a vertex tp, the MSF is obtained after removing 
the vertex r. Prim’s algorithm can be used for building the 
MSF (see Algorithm 1) [39], The efficient implementation of 
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Algorithm 1 Prim’s Algorithm 
Require: Connected graph G = (V, E, W) 

Ensure: Tree T* = (K*, £*, W’) 

V* = {f}, is an arbitrary vertex from V 

while V* f- V do 

Choose edge e.;j € £ with minimal weight such that 

i € 1/* and j 4 V * 

V‘ = V' U {;} 

E* = E* U {e^} 

end while 


the algorithm using a binasy min-heap is possible [40], the 
resulting time complexity of the algorithm is 0(\E\log\V\), 
Finally, a spectral-spatial classification map is obtained by 
assigning the class of each marker to all the pixels grown 
from this marker. 

IV. EXPERIMENTAL RESULTS AND DISCUSSION 

Two different hyperspectral images were used for the ex- 
periments, with different contexts (one urban area and one 
agricultural area) and recorded by different sensors (ROS1S 
and AVIRIS airborne imaging spectrometers). These data sets 
and the corresponding results are discussed in the next two 
sections. 

A. Classification of the University of Pavia Image 

The University of Pavia image was recorded by the ROS1S 
optical sensor over the urban area of the University of Pavia, 
Italy. The image is 610 x 340 pixels, with a spatial resolution 
of 1,3 m/pixel. The number of data channels in the acquired 
image is 1 1 5 (with a spectral range from 0.43 to 0.86 pm). The 
12 most noisy channels have been removed, and the remaining 
103 bands were used for the experiments. Nine classes of 
interest are considered, which are detailed in Table I. Figure 5 
shows a three-band false color image and the reference data. 
The training and test sets are composed of 3921 and 40002 
pixels, respectively. More information about the image, with 
the number of training and test samples for each class can be 
found in [14], 

The segmentation of the considered image was performed, 
using the three different techniques discussed in the previous 
section. For the EM algorithm, the maximum number of 
clusters was chosen equal to 1 0 (typically slightly superior to 
the number of classes). Before applying the EM technique, a 
feature extraction on the original 103-band image was applied, 
using the method of PCFA [31] to get a 10-band image Y up- 
Pixels from the training set were used for selecting features. 
The method produced an averaging of the following groups 
of adjacent spectral channels: 1-4, 5-10, 11-24, 25-35, 36-43, 
44-68, 69-72, 73-75, 76-79, 80-103. 

For the RHSEG algorithm we chose S wg kt — 0.1 since 
the image of this urban area contains classes with mostly 
dissimilar spectral responses. A segmentation map at an ap- 
propriate level of segmentation detail was chosen interactively 
with the program HSEGViewer. The obtained watershed. EM 


and RHSEG segmentation maps contained 1 1802, 22549 and 
7575 regions, respectively. 

The multiclass pairwise SVM classification, with the Gaus- 
sian Radial Basis Function (RBF) kernel, of the original image 
was performed, with the parameters chosen by fivefold cross 
validation: C — 128, 7 - 0.125. The results of the pixelwise 
classification were combined with the segmentation results, 
using the majority' voting approach. Finally, the marker selec- 
tion (see Figure 5(g); 132521, i,e., 64% of pixels were selected 
as markers) and the construction of an MSF were performed, 
resulting in the MSSC-MSF spectral-spatial classification map 
depicted in Figure 5(h). 

Table I summarizes the global and class-specific accuracies 
of the pixelwise SVM, segmentation plus majority voting 
( WH+MV , EM+MV . , RHSEG+MF for three segmentation 
techniques, respectively) and the proposed MSSC-MSF clas- 
sification methods. The following measures of accuracy were 
used: Overall Accuracy (OA is the percentage of correctly 
classified pixels). Average Accuracy (AA is the mean of class- 
specific accuracies, i.e., the percentage of correctly classified 
pixels for each class) and kappa coefficient (k is the per- 
centage of agreement, i.e., correctly classified pixels, corrected 
by the number of agreements that would be expected purely 
by chance [41]). In order to compare performances of the 
proposed technique with the previously proposed methods, 
we have also included results of the well-known ECHO 
spatial classifier [7], as well as the results obtained using the 
construction of an MSF from the probabilistic SVM-derived 
markers followed by majority voting within connected regions 
(SVMMSF+MV) [17], 

In addition, we assessed the importance of spectral-spatial 
approaches for marker selection. For this purpose we replaced 
the WH+MV, EM+MV, RHSEG+MV classification maps by 
three maps obtained using standard pixelwise classification 
techniques (we call this modified scheme an MC-MSF clas- 
sification method). SVM, Maximum Likelihood (ML) and 
3-Nearest Neighborhood (3-NN, using the SAM distance) 
methods were used for this purpose. The ML and the 3- 
NN techniques were applied on the 10-band image Yyp 
feature vectors. The accuracies of the modified MC-MSF 
classification, as well as pixelwise classification results are 
given in Table I. 

As can be seen from Table I, the SVM method gives 
the highest accuracies among all the pixelwise classification 
techniques. All the spectral-spatial approaches yield higher 
classification accuracies when compared to pixelwise methods. 
The proposed MC approach for marker selection improves 
accuracies, when compared to those obtained by classification 
techniques used in the preliminary step of the marker selection 
procedure, both for the MC-MSF and MSSC-MSF methods. 
The best global and the best class-specific accuracies for most 
classes are achieved by applying the proposed MSSC-MSF 
method. According to the results of the McNemar’s test [42], 
the MSSC-MSF classification map is significantly more accu- 
rate when compared to those obtained by other classification 
approaches, using 5% level of significance. In this case, the 
overall accuracy is improved by 1 6.9 percentage points and the 
average accuracy is improved by 10.3 percentage points, when 






Fig. 5. University of Pavia image, (a) Three-band color composite, (b) Reference data; ; . meadows, gravel, trees, metal sheets, bare soil, bitumen, 

bricks and . (c) SVM pixehvise classification map. (d) ECHO classification map. (e) S'VMMSF+MV classification map. (f) MC-MSF classification 

map. (g) MSSC-MSF marker map. (h) MSSC-MSF classification map. 


TABLE I 

Classification Accuracies in percentage for the University of Pavia Image: Overall Accuracy (OA), Average Accuracy (AA), Kappa 

Coefficient (k) and Class-Specific Accuracies. 
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Trees 

Metal sheets 
Bare soil 
Bitumen 
Bricks 
Shadows 
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compared to the SVTV1 classification. All the class-specific 
accuracies are higher than 96%. Only the accuracy for the 
class shadows, representing small spatial structures, is slightly 
reduced when compared to the SVM results (the drawback of 
applying spectral-spatial classification to small structures was 
discussed for instance in [14]). The MSSC-MSF classification 
accuracies are much higher than the MC-MSF accuracies. 
Furthermore, the presented classification accuracies are higher 
than all previous results we have found in the literature for the 
same data. 

Figure 5 depicts the MC-MSF and MSSC-MSF classification 
maps, as well as the SVM, ECHO and SVMMSF+MV classi- 
fication maps given for comparison. In Figure 5(g) [ MSSC- 
MSF marker map], it can be seen that the marker pixels, 
i.e., the most reliable classified pixels, are typically located 
at the center of spatial structures, while borders of structures 
are under a high risk of being misclassified. The MSSC- 
MSF classification map [see Figure 5(h)] contains much more 
homogeneous regions when compared to the maps obtained by 
other pixelwise and spectral-spatiat approaches. These results 
prove the importance of the use of MC systems and spatial 
information throughout the classification procedure. 

B. Classification of the Indian Pines Image 

The proposed scheme was also tested on the Indian Pines 
image of a vegetation area, acquired by the AVIRIS sensor 
over the Indian Pines site in Northwestern Indiana. The image 
has spatial dimensions of 145 by 145 pixels, with a spatial 
resolution of 20 m/pixel. Twenty water absorption bands (104- 
108, 150-163, 220) have been removed [18], and a 200- 
band image was used for the experiments. The reference data 
contain sixteen classes of interest, which represent mostly 
different types of crops and are detailed in Table II. A three- 
band false color image and the reference data are presented 
in Figure 6 . We have randomly chosen 50 samples for each 
class from the reference data as training samples, except for 
classes “ alfalfa ”, "grass/pasture-mowed” and “oats". These 
classes contain a small number of samples in the reference 
data. Therefore, only 15 samples for each of these classes 
were chosen randomly to be used as training samples. The 
remaining samples composed the test set. 

Segmentation of the Indian Pines image was performed, 
using the three discussed techniques. For the EM technique, 
the upper bound on the number of classes was chosen equal to 
17, and a feature reduction has been previously applied, using 
the method of PCFA [31] to get a 10-band image Y /,v. The 
following groups of bands were averaged: 1-18, 19-36, 37-53, 
54-57, 58-61, 62-75, 76-81, 82-99, 100-140, 141-200. 

Since some classes have very similar spectral responses in 
the Indian Pines image (for instance, three classes of com and 
three classes of soybeans), we set S wg f tt — 0.0 for the RHSEG 
method. A segmentation map at the relevant level of hierarchy 
was chosen with the program HSEGViewer, The resulting 
watershed, EM and RHSEG segmentation maps contained 
1277, 3832 and 823 regions, respectively. 

A pixelwise classification on the 200-band image was 
performed, using the multiclass one versus one SVM classifier 



(e) (f) 


Fig. 6. Indian Pines image, (a) Throe-band color composite, (b) Reference 
data; Corn-no ti!i, Corti-min tilt. Com, Soybeans-no till, Soybeans -min till, 
Soyboans-clean till, , Grass, 'pasture, Grass/trecs, Grass/pasture-rriowed, 
Hat - windrowed, Wheat, Woods, Bidg'Grass- Tree-Drive h 

. fc) SVM pixelwise classification map. (d) SVMMSF+MV classification 
map. (c) MC-MSF classification map. (?) MSSC'-MSF classification map, 

with the Gaussian RBF kernel. The optimal parameters C 
and 7 were chosen by fivefold cross validation: C = 128, 
7 ~ 2 fl . After the segmentation results were combined with 
the pixelwise classification map, the marker selection (14409, 
i.e., 68 % of pixels were selected as markers) and the MSF 
construction were applied, as described in the previous section. 

Table 11 gives the global and class-specific accuracies of 
the pixelwise SVM, segmentation followed by majority voting 
and the proposed MSSC-MSF classification techniques. The 
performances of the proposed approach are compared with 
those obtained by the ECHO and SVMMSF+MV methods, 
as described in the previous section. Finally, the MC-MSF 
classification was applied in the same was as for the previous 
data set. 

From the table, the similar conclusions as for the previous 
data set can be derived. The SVM classification yields the 
best accuracies among all the applied pixelwise methods. 
Spectral-spatial classification accuracies are always higher 
when compared to pixelwise accuracies. The proposed MC 
method succeeds in combining several classification results 






TABLE II 

Classification Accuracies in Percentage for the Indian Pines Image: Overall Accuracy (OA), Average Accuracy (AA), Kappa 

Coefficient (k) and Class-Specific Accuracies. 



3-NN 

ML 

SVM 

ECHO 

WH+MV 

EM+MV 

RHSEG 

+MV 

SVMMSF 

+MV 

MC- 

MSF 

MSSC- 

MSF 

OA 

66.27 

75.41 

78.17 

82.64 

86.63 

83.60 

90.86 

91.80 

86.66 

92.32 

AA 

76,77 

79.61 

85.97 

83.75 

91.61 

85.34 

93.96 

94.28 

92.58 

94.22 

K 

62.04 

72.25 

75.33 

80.38 

84.83 

81.43 

89.56 

90.64 

84.82 

91.19 

Com-no till 

41,84 

71.39 

78.18 

83.45 

94.22 

89.09 

90.46 

93.21 

83.82 

89.74 

Com-min till 

62.24 

63.01 

69.64 

75.13 

78.06 

75.64 

83.04 

96.56 

74.62 

86.99 

Com 

73.37 

85.87 

91.85 

92.39 

88.59 

65.22 

95.65 

95.65 

96.74 

95.11 

Soybeans-no till 

67.43 

79,43 

82.03 

90.10 

96.30 

88.14 

92.06 

93.91 

93.36 

91.84 

Soybeans-min till 

53.91 

52.65 

58.95 

64.14 

68.82 

65.67 

84.04 

81.97 

72.91 

89.16 

Soybeans-clean till 

64.72 

85.99 

87.94 

89.89 

90.78 

95.04 

95.39 

97.16 

95.92 

97.34 

Alfalfa 

84.62 

48.72 

74.36 

48.72 

94.87 

94.87 

92.31 

94.87 

94.87 

94.87 

Grass/pasture 

86.35 

93.51 

92.17 

94.18 

95.08 

93.96 

94.41 

94.63 

98.21 

94.63 

Grass/trees 

91.97 

94.69 

91.68 

96.27 

97.99 

96.41 

97.56 

97.27 

97.70 

97.85 

Grass/pasture-mowed 

100 

36.36 

too 

36.36 

too 

too 

100 

100 

100 

100 

Hay-windrowed 

95.67 

97.72 

97.72 

97.72 

99.54 

99.32 

99.54 

99.77 

99.54 

99.77 

Oats 

80,00 

100 

100 

100 

100 

40.00 

100 

too 

100 

100 

Wheat 

99.38 

98.15 

98.77 

98.15 

99.38 

98.77 

98.15 

99.38 

99.38 

99.38 

Woods 

86.17 

95.42 

93.01 

94.21 

97.11 

96.70 

98.63 

99.68 

98.47 

99.44 

Bldg-Grass-Tree-Drives 

45.15 

73.03 

61.52 

81.52 

69.39 

66.67 

82.12 

68.79 

77.88 

73.64 

Stone-steel towers 

95.56 

97.78 

97.78 

97.78 

95.56 

100 

100 

95.56 

97.78 

97,78 


for further improvement of accuracies. The MSSC-MSF yields 
the best OA, kappa coefficient and most of the class-specific 
accuracies. The AA is only slightly (non-significantiy) lower 
when compared to that obtained by the recently proposed 
SVMMSF+MV method. Following the results of the McNe- 
rnar’s test, the MSSC-MSF and SVMMSF+MV accuracies are 
not significantly different, using 5% level of significance. 
These two techniques significantly outperform other classifi- 
cation approaches. 

Figure 6 shows the SVM, MC-MSF , MSSC-MSF and 
SVMMSF+MV classification maps. As can be seen, the MSSC- 
MSF map contains much more homogeneous spatial struc- 
tures, when compared to the SVM and MC-MSF maps, and 
is comparable with the SVMMSF+MV map. Again, spectral- 
spatial marker-based techniques yielded the most accurate 
classification maps. 

Although for the Indian Pines image, the MSSC-MSF and 
SVMMSF+MV methods yield similar results, here we stress 
the advantages of the proposed MSSC-MSF approach versus 
the previous one for spectral-spatial classification: 

• Robustness: While for the SVMMSF+MV method the 
marker selection strongly depends on the performances 
of the selected piselwise classifier, the MC approach mit- 
igates this dependence. Since in the MSSC-MSF scheme, 
different segmentation maps are combined with one pix- 
elwise classification map, the choice of the classifier is 
also important. However, if in the SVMMSF+MV method 
a pixel was wrongly classified with a high probability, 
it will yield a wrong marker. In the new approach, the 
majority voting within segmentation regions can correct 
the misclassification result for a particular pixel, before 
the marker map is built. 

. Computational Complexity: In the SVMMSF+MV 

method, the probabilistic pixelwise SVM classification 
pail is the most time-consuming [43], In the MSSC-MSF 


approach, SVM classification is performed without the 
computation of probability estimates; this reduces the 
pixelwise classification part execution time. The unsu- 
pervised segmentation techniques are much less time- 
consuming when compared to the SVM classification. 
Furthermore, their efficient implementations arc available, 
and they can be executed in parallel at the same time 
with the SVM classification. As a conclusion, the efficient 
implementation of the proposed MSSC-MSF approach 
is possible, which would run faster than the previously 
proposed MSSC-MSF method. 

V. Conclusions 

Hyperspectral sensors capture images in hundreds of narrow 
spectral channels. The detailed spectral signatures for each 
spatial location provide rich information about an image scene, 
leading to better discrimination between physical materials 
and objects. However, interpretation of these high-dimensional 
signatures is a challenging task. Although pixelwise classi- 
fication techniques have given high classification accuracies 
when dealing with hyperspectral data, the incorporation of 
tire spatial context into classification procedures yields further 
improvement of tire accuracies. 

In this paper, a new method for spectral-spatial classifi- 
cation of hyperspectral images based on multiple classifiers 
is proposed. First, a marker map is constructed by selecting 
the pixels assigned by several spectral -spatial classifiers to the 
same class. This ensures a robust and reliable selection. Then, 
an MSF rooted on the selected markers is built. Experimental 
results demonstrated that the proposed method improves clas- 
sification accuracies, when compared to previously proposed 
classification schemes, and provides classification maps with 
homogeneous regions. 

In conclusion, the proposed methodology succeeded in 
taking advantage of multiple classifiers and the spatial and the 
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spectra! information simultaneously for accurate hyperspectral 
image classification. The method yields accurate results for 
different data sets, i.e., data containing large spatial structures 
and/or small and complex structures, with spectrally dissim- 
ilar and/or spectrally confusing classes. Furthermore, its the 
efficient implementation is possible. 

While performing especially well for classification of ho- 
mogeneous regions, the proposed approach has a drawback 
common to most of spectral-spatial techniques: It produces 
a smoother classification map when compared to pixelwise 
ones, and therefore it risks impairing results near the borders 
between regions (where mixed pixels 4 are often encountered) 
or in textured areas. Spectral unmixing techniques [44] can be 
used for accurate analysis of region borders, while segmenta- 
tion in the sense of texture [45] can be applied for textured 
regions. 

In the future, we wilt further explore the integration of 
spectral-spatial approaches in MC systems for accurate and ro- 
bust classification of hyperspectral images. Since the incorpo- 
ration of the spatial information in classification significantly 
improves accuracies, it is of interest to further investigate per- 
formances of the proposed spectral-spatial approaches when a 
very limited number of training samples is available. 
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